Skip to content

[FEATURE] Add ABot-World-0-5B-LF single-GPU interactive pipeline - #26

Open
youngmagician114514 wants to merge 2 commits into
Tele-AI:mainfrom
youngmagician114514:feat/abot-world-single-gpu
Open

[FEATURE] Add ABot-World-0-5B-LF single-GPU interactive pipeline#26
youngmagician114514 wants to merge 2 commits into
Tele-AI:mainfrom
youngmagician114514:feat/abot-world-single-gpu

Conversation

@youngmagician114514

Copy link
Copy Markdown

Description

Add a native TeleFuser single-GPU integration for the official
ABot-World-0-5B-LF checkpoint, including causal interactive generation and
a direct browser controller.

Motivation

TeleFuser did not provide an ABot-World interactive pipeline. This change
enables image-conditioned, action-controlled world rollout while retaining
causal KV, scheduler, RNG, and VAE temporal decode state across control blocks.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Performance improvement
  • Code refactoring
  • Documentation update
  • Other

Changes Made

  • Add ABotWorldDiT, ABotWorldDenoisingStage, and TeleFuser pipeline classes.
  • Add persistent single-session interactive generation.
  • Add six sink latents and a twelve-latent rolling KV window.
  • Add bounded RoPE positions for long-horizon sessions.
  • Add direct HTTP browser demo with ordered 12 FPS playback and FIFO backpressure.
  • Add unit, checkpoint-contract, and 30-block GPU smoke tests.

Testing

  • ABot-World unit tests pass
  • Manual browser testing performed
  • Benchmarks added/updated

Test commands:

python -m pytest \
  tests/unit/pipelines/abot_world \
  tests/unit/models/test_wan22_video_vae.py \
  -q
ABOT_WORLD_MODEL_ROOT=/path/to/ABot-World-0-5B-LF \
python -m pytest -m filesystem \
  tests/integration/test_abot_world_checkpoint.py \
  -v
CUDA_VISIBLE_DEVICES=0 \
ABOT_WORLD_MODEL_ROOT=/path/to/ABot-World-0-5B-LF \
ABOT_WORLD_TEST_IMAGE=/path/to/initial.png \
python -m pytest -m "gpu and slow" \
  tests/integration/test_abot_world_smoke.py \
  -v -s

Checklist

  • Code follows the project's coding standards (ruff)
  • Pre-commit hooks pass (pre-commit run --all-files)
  • All repository tests pass (pytest tests/)
  • New tests added for new functionality
  • Documentation updated
  • Commit messages are clear and descriptive
  • PR title follows the convention: [TYPE] Brief description

Related Issues

N/A

Additional Notes

  • Scope is limited to one GPU and one retained interactive session.
  • Transport is direct HTTP; LiveKit, WebRTC, TURN, and stream-serve are not introduced.
  • The six-sink KV policy and bounded RoPE positions are intentional stability behavior.
  • Full repository-wide pytest tests/ and pre-commit hooks were not run.

GPU Architecture Support

  • SM80 (Ampere, Ada Lovelace)
  • SM90 (Hopper H100)
  • SM100+ (Blackwell)

No new architecture-specific CUDA kernels are added.

Performance Impact

No formal benchmark is included. The runtime preloads model weights, retains
session state, consumes frames at 12 FPS, and applies producer backpressure
instead of dropping frames during normal playback.

@lzx1413

lzx1413 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

How about serving this world model as the lingbot world with livekit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants